Analyzing CVE-2017-3731: Truncated Packets Can Cause Denial of Service in OpenSSL

OpenSSL is a popular open-source library for SSL and is used by various software and companies across the world. In January, OpenSSL released an update that fixed multiple vulnerabilities. One of them is CVE-2017-3731, which can cause a denial of service due to a crash. McAfee Labs analyzed this vulnerability to provide detection for customers. 

Figuring out the changes using patch diff

The patch modified a couple of files related to various cipher algorithms. For this report we will examine e_chacha20_poly1305.c. The following code shows the patch to this file, taken from https://git.openssl.org/?p=openssl.git;a=commitdiff;h=2198b3a55de681e1f3c23edb0586afe13f438051.

We can see that a simple step was added to check the value of variable length against the constant POLY1305_BLOCK_SIZE and just below that we see that this constant is subtracted from the variable “len.”

If we look at the declaration, POLY1305_BLOCK_SIZE is declared in the file poly1305.h as “#define POLY1305_BLOCK_SIZE 16.” The variable len is defined in e_chacha20_poly1305.c as “unsigned int len;”

So if the variable len is less than 16, it will cause an integer underflow, that is, the value of len will become very large. When used, this value can cause problems with the normal program flow because the value of len will be incorrect.

Digging further

We can see in the preceding image that this len value is assigned to “actx->tls_payload_length.” Then the function chacha20_poly1305_cipher is called. Inside this function actx->tls_payload_length is assigned to the variable “plen”:

Notice that variable plen will now have the very large value that we got from the previous len integer underflow. We can further see that the value of plen is passed to the function poly1305_Update:

Poly1305_Update will carry this large value as it calls the function Poly1305_blocks:

If we take a closer look at the function, we can see that the variable len contains a very large integer value, which is used as the counter in a “while” loop:

We can also see a call to the function U8TOU32, which reads the value of *inp (a pointer), and that the value of *inp is increased by POLY1305_BLOCK_SIZE for each iteration of the loop. Because the value of len is very large, eventually *inp will point to nonreadable memory. Attempting to read that will cause an access violation error—resulting in an OpenSSL crash.

Exploiting the vulnerability from the network

To exploit this vulnerability, a client needs to use the chacha20_poly1305 cipher suite (or another vulnerable cipher, as can be seen from patch diff) and send an encrypted handshake message in which the record length is less than 16 bytes (in the case of chacha20_poly1305 cipher). This will cause an integer underflow and OpenSSL will crash, as we see in the following images running OpenSSL and Gnu Debugger:

Conclusion

OpenSSL is very popular and thus can be a target for denial of service attacks. These types of vulnerabilities can impact many installations. We recommend that users update their OpenSSL installations to the latest version.

McAfee Network Security Platform customers are protected against this vulnerability through signature ID: 0x45c09400.

Introducing McAfee+

Identity theft protection and privacy for your digital life

Download McAfee+ Now